Maximizing Multimodal: Exploring the search frontier of text-image models to improve visual find-ability for creatives
Nathan Day • Location: Theater 5 • Back to Haystack 2025
“Objective: Describe where and how we have improved the search experience in our product with open source multi-modal models and libraries. Real world examples from the things we have shipped (and decided not to ship) to production, including AB test results of our relevancy changes.
Outline:
- Cover the architecture of our open source hybrid search stack at Eezy (Elasticsearch, FAISS, PyTorch models)
- Describe the capabilities and limitations of openCLIP (and vector embeddings at large) for retrieval tasks and current pain points and work arounds we have engineered.
- Highlight meaningful stops on our product roadmap from the last 2 years of deploying features into production.
- Describe notable missteps and surprises uncovered along the way, so people see it’s not all roses in the AI powered future.
- Demo BORGES, a novel search framework that allows users to search with multiple queries in multiple modalities for a nuanced navigation of the catalog to find exactly what they need
Audience:
- Anyone curious about real-world results we have extracted from AI
- Search practitioners developing hybrid search applications
- PyTorch and transformers enthusiasts interested in applications in vector space”
Nathan Day
Eezy